Integrating human- and computer-based approaches to feature extraction and analysis

A major goal of imaging, graphics, and visualization systems is to help doctors, scientists, engineers, and analysts identify patterns and features in complex data. Some feature-extraction system use algorithms to automatically identify features. Other methods use human observers to identify features visually, or to discover features by interactively manipulating visual representations. Although automatic feature-extraction algorithms are often directed by human observation, and human pattern recognition is often supported by algorithmic tools, very little work has been done to explore how to capitalize on the interaction between human and machine pattern recognition. This paper introduces a preliminary roadmap for guiding research in this space. One key concept is the explicit consideration of the task, which determines which methods and tools will be most effective. The second is the explicit inclusion of a “human-in-the-loop,” who interacts with the data, the algorithms, and representations, to identify meaningful features. The third is the inclusion of a process for creating a mathematical representation of the features that have been “carved out” by the human analyst, for use in comparison, database query and analysis.